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(57) Abstract 

An automatic natural translation system for translating input text in a source natuial language such as En^ish. to output text in 
a target natural language, such as Japanese, as the input text is being generated, includes a translation activator that detennlnes when a 
pause occurs in the creation of source language input text in a document or file, and effects, in response to the pause, a translation of tiic 
input text in a target language, up to that point that the pause was sensed. The translation system can fuitfaer effect a translation as text 
is being generated, in response to sensing a certain text structure or in response to an Input function. Each time an automatic translaticm 
is performed, the translation can begin at a starling point, such as the beginning of a word processing document into which the text was 
entered, the beginning of immediately received input text, or the beginning of a page or paragraph, thus allowing any changes fai sentence 
structure or recent edits to the Input text to be reflected in the output text translation. 
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AUTOMATIC TRANSLATION AND RETRANSLATION SYSTEM 

<^ ,-^T fi ??-^''fc^"t-.ete R «»lnfpri Applirations and Patents 
This is a continuation-in-part of international patent applications PCT/US96/05567, 
PCT/US96/10283, and PCT/US97/ 10005 which were filed with the U.S. Receiving Office on 
April 23, 1996, June 14, 1996, and June 9, 1997 respectively, via the Patent Cooperation Treaty 
5 (PCT) designating Japan and the United States, all of which are incorporated herein by reference. 
Also U.S. Patent No. 5,528,491 is a related patent, the entire contents of which are incorporated 
herein by reference. 

T<.r.hnicfll Field 

The invention relates to automated natural language translation, particularly, a natural 
1 0 language translation system in which a document is translated automatically as it is being created 
by an author, user or system. 

BsHlKgrftl""^ Tfifnrmation 
Various schemes for the machine-based translation of natural language have been 
proposed. Typically, a system used for translation includes a computer which receives input in 
1 5 one natural language and perfonns operations to supply output in another natural language. This 
type of translation has been an inexact and lengtiiy one, and tiie resulting output often requires 
significant editing by a skilled operator. 

The translation operation performed by known systems generally includes a structural 
conversion operation. The objective of structural conversion is to transform a given parse tree 
20 (i.e., a syntactic structure tree) of the source language sentence to the corresponding tree in the 
target language. Two types of structural conversion have been tiried, grammar-rule-based and 

template-to-template. 

In grammar-rule-based structural conversion, tiie domain of structural conversion is 
limited to the domain of grammar rules that have been used to obtain the source-language parse 
25 tree (i.e., to a set of subnodes tiiat are immediate daughters of a given node). For example, given 

VP = VTO 1 + NP (a VerbPhrase consists of a SingleObject 
Transitive Verb and a NounPhrase. in that 
order) 
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and 

Japanese: 1 + 2 => 2 + 1 (Reverse the order of VTOl and NP), 
each source-language parse tree that involves application of the rule is structurally converted in 
such a way that the order of the verb and the object is reversed because the verb appears to the 
right of its object in Japanese. This method is very efiFicient in that it is easy to find out where 
the specified conversion applies; it applies exactly at the location where the rule has been used to 
obtain the source-language parse tree. On the other hand, it can be a weak conversion 
mechanism in that its domain, as specified above, may be extremely limited, and in that natural 
language may require conversion rules that straddle over nodes that are not siblings. 

In template-to-template structural conversion, structural conversion is specified in terms 
of input/output (I/O) templates or subtrees. If a given input template matches a given structure 
tree, that portion of the structure tree that is matched by the template is changed as specified by 
the corresponding output template. This is a very powerful conversion mechanism, but it can be 
costly in that it can take a long period of time to find out if a given input template matches any 
portion of a given structure tree. 

Notwithstanding the type of conversion carried out by conventional translation systems, 
such systems only facilitate text translation in the presence of a direct translation command from 
a user or operator after the user or operator has completed typing into the system a document. As 
a resuh, these systems do not lend to language learning or allow the user or operator to examme 
the translation on a word-per-word or sentence-per-sentence basis, and thus, variations in the 
translation of words in different contexts are more difficult for the user or operator to detect. 

Summary nf thet Invention 
The automated natural language translation system according to the present invention has 
many advantages over known machine-based translators. After the system of the invention 
automatically selects the best possible translation of the input textual infonnation and provides 
the user with an output, such as. for example, a Japanese language translation of English- 
language input text, the user can then interfece with the system to edit the displayed translation 
or to obtain alternative translations in an automated fashion. An operator of the automated 
natural language translation system of the invention can be more productive because the system 
allows the operator to retain just the portion of the translation that he or she deems acceptable 
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while causing the remaining portion to be retranslated automatically. Since this selective 
retranslation operation is precisely directed at portions that require retranslation, operators are 
saved the time and tedium of considering potentially large numbers of incorrect, but highly 
ranked translations. Furthermore, because the system allows for arbitrary granularity in 
translation adjustments, more of the final structure of the translation will usually have been 
generated by the system. TTie system thus reduces the potential for human (operator) error and 
saves time in edits that may involve structural, accord, and tense changes. The system efficiently 
gives operators the fiiU benefit of its extensive and reliable knowledge of grammar and spelling. 

The automated natural language translation system's versatile handling of ambiguous 
sentence boundaries in the source language, and its powerful semantic propagation provide 
further accuracy and reduced operator editing of translations. Stored statistical information also 
improves the accuracy of translations by tailoring the preferred translation to the specific user 
site. The system's idiom handling method is advantageous in that it allows sentences that 
happen to include the sequence of words making up the idiom, without intending the meaning of 
the idiom, to be correctly translated. "Die system is efficient but still has versatile fimctions such 
as long distance feature matching. The system's structural balance expert and coordinate 
structure expert effectively distinguish between intended parses and unintended parses. A 
capitalization expert effectively obtains correct interpretations of capitalized words in sentences, 
and a capitalized sequence procedure effectively deals with multiple-word proper names, without 
completely ignoring common noun interpretations. 

In one aspect, the invention is directed to an improvement of the automated natural 
language translation system, wherein the improvement relates to automatically translating a 
source natural language in a document or file as the input text is being generated, such as. for 
example, English, into output text in target natural language, such as, for example, Japanese, 
Spanish, or other natural language. In one aspect, the natural language translation system causes 
a translation to occur when a pause is sensed during the entry of input text, such as a pause in an 
input character stream. In another aspect, the natural language translation system can 
automatically translate the recently received input character stream along with input text that has 
previously been translated, causing a retranslation of certain of the input text. In another aspect, 
the natural language translaUon system can automatically translate input text that has caused a 
modification or change to the previously received input text. 
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In another aspect of the invention, the automatic translation can commence at a starting 
• point in the input text, such as, for example, at the beginning of a document, paragraph, word or 
other location, thus allowing any changes in sentence structure or edits to be absorbed into and 
reflected in the final translation. 
5 In another aspect of the invention, the automatic translation can commence at a point in 

the input text at which a modification to previously received input text was made, such as for 
example, in the middle of a paragraph that was previously received but recently edited. 

In another aspect, the invention relates to an improvement of the automatic global natural 
language translation system, wherein the improvement relates to automatically translating and 
1 0 retranslating input text in a source natural language such as, for example. English, into output 
text in a target natural language, such as, for example, Japanese. Spanish, or other natural 
language, after certain text structure has been generated, such as, for example, a sentence 
fragment. 

In another aspect, the invention relates to an improvement of the automatic natural 
1 5 language translation system, wherein the improvement relates to automatically translating and 
retranslating input text in a source natural language such as, for example. English, into output 
text in target natural language, such as, for example, Japanese, Spanish, or other natural 
language, when a user, operator oi system sends a signal conveying the depression of a 
touchscreen or an enter key on a keyboard, a mouse click on an icon, or other input signal, such 
20 as a function signal not producing a character stream. 

In yet another aspect, the automatic natural language translation system can effect a 
translation automaticaUy in response to a pause in the input character stream by using a counter 
that begins counting as the pause is sensed. In another embodiment of the invention, the 
automatic natural language translation system can effect a translation automatically as a certain 
25 text structure, or as an input signal relating to a system function is sensed. 

In another aspect of the invention, the automatic translation system of the present 
invention can be used as a language learning tool, by translating the input text immediately as the 
input text is being generated and displaying the translated output text along side the input text as 

the translation is occurring. 
30 The foregoing and other objects, aspects, features, and advantages of the invention will 

become more apparent from the following description and from the claims. 
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»P^f r^«r."ptinn of the Drawings 
In the drawings, like reference characters generally refer to the same parts throughout the 
different views. Also, the drawings are not necessarily to scale, emphasis instead generally being 
placed upon illustrating the principles of the invention. 
5 FIG. 1 is a block diagram illustrating a system for performing automated translation of 

natural language. 

FIG. 2 is a data flow diagram illustrating overall functioning of the system of FIG. 1. 

FIG. 3 is a flow diagram illustrating the operation of the system of FIG. 1. 

FIG. 4 is a flow diagram illustrating the operation of the end-of-sentence function of the 

1 0 preparser of the system of FIG. 1 . 

FIG. 5 is a flow diagram illustrating the operation of the parser of the system of FIG. 1 . 
FIG. 6 is a flow diagram illustrating the semantic propagation operations of the system of 

FIG. 1. 

FIG. 7 is a flow digram illustrating the stmctural conversion operations of the system of 

15 FIG. 1. 

FIG. 8 is a flow diagram illustrating the expert evaluator of the system of FIG. 1 . 
FIG. 9 is a diagram of a sample graph used by the system of FIG. 1 for the exemplary 
phrase **by the bank". 

FIG. 1 OA is a functional block diagram illustrating a system for automatically translating 
20 input text according to one embodiment of the present invention. 

FIG. lOB is a functional block diagram illustrating a system for automatically translating 
input text acoordmg to another embodiment of the present invention. 

FIG. 1 1 is a block diagram illustrating one embodiment of the translation activator in 
accordance with the present invention. 
25 FIG. 12A is a flow diagram illustrating the operation of the translation activator 

according to one embodiment of the present invention. 

FIG. 12B is a flow diagram illustrating the operation of the translation activator 
according to another embodiment of the present invention. 

FIGS. 13A, 13B and 13C illustrate an example of a screen display showing a translation 
30 from a source language to a target language using the system of the present invention. 

FIG. 14 is a flow diagram illustrating the operation of the translation activator according 
to another embodiment of the present invention. 
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FIG. 15 is a flow diagram illustrating the operation of the translation activator according 
to another embodiment of the invention. 

FIG. 16 is a flow diagram illustrating the operation of the translation activator according 
to another embodiment of the invention. 



nptailed nescription 

A general description of an automated natural language translation system according to 
the invention is first provided below without reference to any of the drawings. After the general 
description of the system, reference will be made to the various drawings. 

An automated natural language translation system of the present invention can translate 
from a source natural language to a target natural language. In one embodiment, the system can 
translate from English to Japanese. In another embodiment, the system can translate from 
Japanese to English. In stUl other embodiments, the system can translate from EngUsh to 
Spanish and Spanish to English. In still other embodiments, the system can translate fix>m 
English to a plurality of different languages, or from a non-English language to another non- 
English language. 

The system comprises means for receiving and storing the source natural language, a 
translation engine for creating a translation into the target natural language, means for displaying 
the translation to a user, and means for obtaining for a user and displaying to a user alternative 
translations. In one embodiment of the system, the translation engine includes a tianslation 
activator, a preparser, a parser, a graph maker, an evaluator, a graph scorer, a parse extractor, and 
a structural converter. The translation activator detennines when and/or under what conditions 
the translation should be effected. The preparser examines tiie input text and resolves any 
ambiguities in input sentence boundaries. The preparser then creates and displays the input text 
in a parse chart seeded with dictionary entiies. The parser parses the chart to obtain possible 
syntactic categories for tiie input text. The graph maker produces a graph of the possible 
syntactic interpretations of the input text based on the parse chart. The graph includes nodes and 
subnodes which are associated with possible interpretations of the input text. The evaluator, 
which comprises a series of experts, evaluates the graph of the possible interpretations and adds 
expert weights to the nodes and subnodes of the graph. The graph scorer uses the expert weights 
to score the subnodes, and the grajA scorer then associates the N best scores with each node. 
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The parse extractor assigns a parse tree structure to the preferred interpretation as determined by 
the graph scorer. The structural converter performs a structural conversion operation on the 
parse tree structure to obtain a translation in the target language. 

In the following three paragraphs, a description is provided of how: (a) the graph scorer 
combines the expert weights to compute a final weighted score for each subnode; (b) the graph 
scorer combines the subnode scores to arrive at a final node score; and (c) linguistic information 
is propagated throu^ the tree of nodes and subnodes. 

To compute the final weighted score for each subnode, the graph scorer associates a 
constant value with each subnode. An analysis of the linguistic information associated with each 
subnode determines the subnode score. See, for example. FIG. 8 where a series of expert 
evaliiators examine the linguistic information stored at each node and subnode. The graph scorer 
adds together the individual weighted scores for each expert to obtain a final weighted average 
for a particular node or subnode. The combination of a plurality of weighted scores into a single 
weighted average score is a standard problem in computer science. One method that can be used 
is to multiply each expert result by a constant number (weight) assigned to that expert The 
weight assigned to each expert is a matter of design choice. The designer can dioose the priority 
(weight) to assign each expert. The" weighted average is the summation of a series of numbers 
wherein each number is multiplied by a constant. For example, 

weighted average = (w,)(x,) + (wjXxj) + ■•• + (wj(xj 

where the weights, w„ wj, ... w„, are all nonnegative and add up to 1 ; See, for example, Spiegel, 
Tv,.^^ Prohlf m.; »f Proh.hilitv «nH Statistics 76 (McGraw-Hill, Inc. 1975) which discusses 
the use of weighted averages in the context of statistical expectations. 

To combine subnode scores to obtain a final node score, the graph scorer can propagate 
the subnode scores from the bottom of the graph up to the top of the graph. Given the graph, 
wherein each node has a set of N scores, it is possible to determine one or more propagation 
methods. One technique which can be used to propagate the subnode scores is memoization 
which is a type of dynamic-programming used to solve optimization problems. The solution to 
optimization problems can involve many possible values (outcomes). The task is to find the 
optimal value. The algorithm used in optimization solves every subsubproblem just once and 
saves the outcome, thus avoiding the need to recompute the answer every time the 
subsubproblem is encountered. For a more detailed explanation of memoization as applied to 
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optimization problems, see, for example. Gormen et al., Tntro^llC ti on to Alporithms 301-314 
. (McGraw-Hill Book Co. 1990). The method described at pages 301, 302, and 312 of 
^^^„^ ^^.toAianrithms is one method that can be used for propagating subnode scoi« 

information tiarough the graph. 
5 In propagating linguistic information through the tree, the semantic propagation part of 

the system operates to propagate semantic information from smaller constituents to the larger 
constituents that they comprise. Semantic propagation s^lies to the four classes of syntactic 
categories (SEMNP. SEMVP, SEMADJ. and VERB) used in the parsing operation. Before 
semantic propagation can occur, the linguistic information stored at the nodes must be analyzed. 
10 The analysis of the semantic information stored at the nodes is guided by a set of rules that tell 
from examining the noun-like and verb-like constituents in a grammar rule which selectional 
restriction slots of tiie verb-like constittients apply to which noun-like objects. Gerald Gazdar 
discusses in his text M,tnri^| T .Wliap^ Pror^^^inHn Prolog (Addison-Wesley PubKshing Co.. 
1989) a set of rules which can be used to analyze the semantic information stored at tiie nodes in 
1 5 a directed acyclic graph similar to that disclosed in the specification. Gazdar discusses tiie use of 
feature matching to match information on adjacent nodes. Gazdar states that feature matching 
involves equations that say that certain 

features appearing on one node must be identical to the features 
appearing on another. Most current work assumes a prmciple 
20 that is responsible for equating one class of feattire 

specifications as they appear on tiie mother category and the 
daughter which manifests tiie morphology associated witii 
those features. This daughter is known as the "head" of the 

phrase. Most phrases only have a single head. Thus, for 
25 example, a verb phrase inherits the tense of its verb since tiie 

latter is the head of the verb phrase. There is no 
straightforward way of specifying this principle on a grammar- 
wide basis witij tiie notational resources tiiat we have used so 
far but we can stipulate tiie effects of tiie principle on a rule- 
30 by-rule basis quite simply ifwe assume tiiatflie relevant 

features are all to be found on a single branch of tiie DAG. Let 
us call tiie label on tiiis branch head. Then we can wnte a 
typical VP rule as follows: 

VP — > V NP PP 

25 <V head> = <VP head> 

This requires tiiat tiie value of tiie head feature on the V and 
tiiat on tiie mother VP be identical. 
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The rules discussed in Gazdar can be easily adapted for each of the syntactic categories discussed 
herein. The linguistic infonnation assigned to each node using Gazdar's rules can be propagated 
through the tree using memoization techniques. 

Thus, summarizing the previous three paragraphs, the weighted average is one method of 
determining the subnode score, each subnode score can be propagated through the graph using 
known memoization techniques as applied to optimization problems, and the strategy discussed 
in Gazdar's text can be used to analyze the linguistic infonnation stored at each node and this 
linguistic information can be propagated through the parse tree chart using memoization 
techniques. 

The automated natural language translation system can perform automated re-translation 
functions after the initial automatic translation. That is, after the system automatically selects the 
best possible translation of the input textual information and provides the user v^^ith an output 
(preferably a Japanese language translation of the input English text, or a Japanese-to-English 
translation), the user can then interface with the system to edit tiie displayed translation or to 
obtain alternative translations in an automated fashion. 

The automated natural language translation system uses a linguistic model which breaks a 
sentence into substrings. A substring is one or more words which occur in the order specified as 
part of the sentence. For instance, substrings of "TTie man is happy" include "The," "The man," 
"man is happy." "is." and "The man is happy" itself, but not "is man." "man man," and "The is." 

Different linguistic models classify substrings in various ways and in different levels of 
detail. For instance, in "They would like an arrow." "an am,w» is typically classified as a noun 
phrase (NP). Some models would also classify "an arrow" with syntactic features (for instance, 
it is a singular noun phrase), and semantic features (it refere to a weapon). If the phrase is 
ambiguous, there may be two or more ways of classifying it. For instance, "an arrow" can also 
refer to a symbol with an arrow-like shape. When linguistic models provide a method for 
resolving ambiguity, they usually do so by combining smaller units into larger units. When 
evaluating a larger unit, these models consider only a portion of the information contained in the 
larger unit. 

In an exemplary embodiment of the system, the semantic property of "an arrow" (symbol 
vs. weapon) is used in evaluating the verb phrase "like an arrow" in the sentence "They would 
like an arrow." In contrast, if the syntax of the phrase "an arrow" were changed as in "He shot it 
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with an arrow," the semantic property of "an arrow" is not used in evaluating the verb phrase 
"shot it with an aitow." 

For any substring of a sentence interpreted in a single way with regard to a specific 
linguistic model (an interpreted substring), exported properties exist. Exported properties are all 
5 properties used to evaluate the combination of an interpreted substring with other units to form 
larger substrings. An export is an interpreted substring interpreted together with its exported 
properties. Properties that are contained within tiie interpreted substring but not exported are 
called substructures. 

The parser of the system includes a grammar database. The parser finds all possible 
10 interpretations of a sentence using grammatical rules. The grammar database consists of a series 
of context-free phrase structure rules of the form X = Al A2 ... An. X is composed of, or made 
from, Al A2 ... An and is referred to as a higher node of lower nodes (subnodes) Al through An. 

The graph maker of the system graphically represents the many possible intopretations 
of a sentence. Each node of the graph corresponds to an report of some substring. In one 
1 5 embodiment of the system, a single export is represraited by a single node. The grqjh contains 
arcs which emanate from the nodes associated with an export. The arcs represent the 
substructure of the export based on the application of grammar rules. The graph may depict at 
least two types of arcs: ( 1 ) a unary arc which points to a single different export of the same 
substring; (2) a binary arc which includes a pair of pointers which points to two exports, the 
20 substrings of which when concatenated form the substring of the original export. Note that the 
formulation described in (2) assvunes a grammar in Chomsky normal form. 

The graph also includes a single starting export § &om which all portions of the graph 
can be reached by following a series of arcs. The starting export comssponds to the aitire 
sentence. 

25 Multiple arcs emanate from a node if and only if the same export can be composed of one 

or more exports (the pair of pointers in a binary arc is not considered multiple arcs for this 
purpose). Multiple arcs point to a node if and only if that export is a component of multiple 
exports. A node with no arcs projecting from it corresponds to a dictionary entry assigned to the 
substring. 

30 A plurality of linguistic experts assign a numerical score to a set of exports. The 

linguistic experts apply the score to each node of the graph. In one embodiment of the system, a 
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scoring airay (where each element of the array is a weight to multiply by a particular expert's 
score) is a fixed length "M" of floating point numbers for any given sentence. 

The score is evaluated by a scoring module which may be integrated witfi the graph- 
making engine and/or the parser. Scores are computed for all exports that make up a higher 
export. The score for the higher export is computed as the sum of the exports that make up the 
higher level export and the scores of any experts that apply to the combination such as a score 
assigned by the structural balance expert. 

The order in which nodes are visited and scored is a standard depth-first graph-walking 
algorithm. In this algorithm, nodes that have been scored are marked and are not scored again. 
During the scoring process, the scoring module evaluates dictionary entry nodes before 
evaluating any of the higher unit nodes. Each dictionary entry gives rise to a single score. 

Multiple scores result where there are multiple ways of making an export, i.e., k ways of 
making the export result in k possible scores. Multiple scores are handled as follows: 

(1) For a unary rule, each of the k scores of the lower export is added 
to the expert values that apply to the unary rule, and the resulting 
vector of k scores is associated with the parent export. 

(2) For a binary rule, assume that the left child has g scores and the 
right child has h scores. Then a total of g times h scores are computed 
by adding each of the left child's scores to each of the right child's 
scores, and in addition, adding the expert values that apply to the 

binary rule. When g times h exceeds N, only the N best scores are .... 
kept with the parent node. 

(3) When a node's expjort can be created in multiple ways, at most N 
scores are added to that node's score list, the best scores being kept. 

When scoring is complete, the above methods assure that each export has associated with 
its node a set of g scores (g ranging from 1 to U) which represent the g most likely ways (relative 
to the linguistic model) of making the export, including all substructure properties which are not 
represented in the export. In the special case of the root node S. the scoring method gives rise to 
the g most likely ways of making the sentence. 

Each score in each score list described above has an associated pointer. The pointer 
provides information to indicate which score(s) of the score list of lower export(s) were 
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combined to produce the higher level score. By following the respective pointers, the g most 
likely inteipretations of the sentence can be extracted as unambiguous parse trees. 

Further details of the automated natural language translation system will now be 
disclosed with reference to FIGS. 1-9. Various improvements according to the invention are 
described thereafter with reference to FIGS. 10-16. 

Referring to FIGS. 1 and 2, an automated natural language translation system 10 
according to the invention includes an input interface 12, a translation engine 16, storage 18. a 
user input device 22, a display 20, and an output interface 14. The input interface 12 is 
constructed to receive a sequence of text in a source language, such as, for example. English or 
Japanese. The input interface 12 may comprise a keyboard, a voice interface, or a digital 
electronic interface, such as a modem or a serial input. The translation engine 1 6 performs 
translation operations on tiie source text, in conjunction with data in storage. The translation 
engine 16 may be comprised entirely of hardwired logic circuitry, or it may contain one or more 
processing units and associated stored instructions. 

The translation engine 16 may include the following elements, or parts of them: A 
translation activator 21 , a preparser 24, a parser 26, a graph maker 28, a parse/translation 
evaluator 30, a parse extractor 32, a structural converter 34, and a user interface 42, which 
includes an alternate parse system 37. The structural converter may comprise a grammar rule 
controlled structural converter 36, a lexicon controlled structural converter 38, and a synthesis 
rule controlled structural converter 40. The storage 1 8 may include one or more areas of disk 
(e.g., hard, floppy, and/or optical) and/or memory (e.g., RAM) storage, or the like. It may store 
the following elements, in whole or in part: a base dictionary 44, technical dictionaries 46, user- 
created dictionaries, grammar rules 48, synthesis rules 50, a semantic feature tree 52, stnicture 
trees 54. and a graph 56. The storage 1 8 also is used to store input textual information in a 
source natural language, output textual information in a target natural language, and all sorts of 
information used or useful in performing tiie translation including one or more dictionaries, 
domain keywords, and grammar rales. The user input interface 22 may comprise a keyboard, a 
mouse, touchscreen, light pen, or other user input device, and is to be used by the operator of tiie 
system. The display may be a computer display, printer or other type of display, or it may 
include otiier means of communicating information to tiie operator. The output interface 1 4 
communicates a final translation of tiie source text in tiie target language, such as Japanese. The 
output interface 14 may comprise a printer, a display, a voice interface, an electronic interface. 
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such as a modem or serial line, or it may include other means for communicating that text to the 
end user. 

In operation of one embodiment of the translation system of the invention, referring to 
FIGS. 1, the translation engine 16 can include translation activator 21, which can be configured 
5 with the remaining elements shown in this figure within the translation engine 16, or as a 

separate component capable of periodically interfacing with the other elements of the translation 
engine 16, as shown in Fig. 1 1 . The translation activator 21 can comprise a computer processing 
unit for determining when a translation of input text should commence. The translation activator 
21 typically sets the parameters the govern this determination, determines, while the text is being 

1 0 generated, whether the conditions surrounding such parameters exist, and initiates the automatic 
translation when such conditions exist. As further described in exemplary embodiments herein, 
the translation activator 21 can generate and store a predetermined time period or value which 
can be used to determine when a lapse in generating the input text exists. The translation 
activator can upon realizing a lapse or pause substantially equivalent to such time period, initiate 

1 5 an automatic translation and/or retranslation, which can be carried out in the translation engine 
16, described herein- Similarly, the translation activator can be configured to process the input 
text and sense the formation of a words, sentences or sentence fragments, and paragraphs, using 
predefined parameters configured to represent each of the above structures, and effect a 
translation when such structures exist. The translation activator and the operation thereof, is 

20 further described herein in Figs. 10-16. 

The preparser 24 first performs a preparsing operation (step 102) on the source text 23, 
which, as further described in Figs. 10-16, can include source text as it is being created, edited, 
received or otherwise compiled in a computer file or document. This operation includes the 
resolution of ambiguities in sentence boundaries in the source text, and results in a parse chart 

25 seeded with dictionary entries 25. The parser 26 then parses the chart produced by the preparser 
(step 104), to obtain a parse chart filled with syntactic possibilities 27. The graph maker 28 
produces a graph of possible interpretations 29 (step 106), based on the parse chart resulting from 
the parsing step. The evaluator 30, which accesses a series of experts 43, evaluates the graph of 
stored interpretations (step 108), and adds expert weights to the graph 31. The graph scorer 33 

30 scores nodes and associates the N (e.g., 20) best scores with each of them 35. The parse extracter 
32 assigns a parse tree structure 39 to this preferred interpretation (step 1 10). The structural 
converter 34, which accesses the conversion tables 58, then performs a structural conversion 
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operation (step 1 12) on the tree to obtain a tianslation 41 in the target language. The user may 
interact with the alternate parse system 37 to obtain alternative translations. 

Referring to FIG. 4, the system begins the preparsing operation by dividing the input 
stream into tokens (step 1 14), which include individual punctuation marks, and groups of letters 
5 that form words. The occurrence of whitespace affects the interpretation of characters at this 
level. For instance, in "x - y" the is a dash, but in "x-y" it is a hyphen. 

The preparser then combines the tokens into words (step 1 16). At this level, it recognizes 
special constructions (e.g-, internet addresses, telephone numbers, and social security numbers) 
as single units. The preparser also uses dictionary lookup to find groupings. For example, if "re- 
1 0 enact" is in the dictionary as "reenact" it will become one word in the sentence, but if it is not, 
then it will remain as three separate *\vords'*. 

The next preparsing phase involves determining where the sentence ends (step 1 1 8). 
During this operation, the preparser accesses the base dictionary and ttie technical dictionaries, 
and any user-created dictionaries, as it follows a sequence of steps for each possible sentence 
1 5 ending point (i.e., after each word of the source text). The preparser need not perform these steps 
in the particular order presented, and these may be implemented as a series of ordered rules or 
they may be hard-coded. 

Referring to FIG. 5, the preparser interprets and records any nonparsable sequence of 

characters, such as a series of dashes: " as a "sentence" by itself, although not one which 

20 will be translated (step 120). The preparser also requires any sequence of two carriage returns in 
a row to be the end of a sentence (step 122). If the first letter of the next word is a lower case 
letter, the preparser will not indicate the end of a sentence (step 124). If a sentence started on a 
new line and is short, the preparser considers it a ''sentence" of its own (e.g., a title). 

The preparser interprets a period, a question mark, or an exclamation mark as the end of a 
25 sentence, except in certain situations involving end parenthesis and end quotes (step 128). In the 
case of sentences that ends with or ?" or the like, the preparser uses virtual punctuation marks 
after the quote in addition to the punctuation before the quote. Altematives for the underlying 
punctuation required for ?" are illustrated in the following examples: 

The question was "What do you want?". 
30 Did he ask the question "What do you want?"? 

Are you concerned about "the other people"? 
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In English, each of these is likely to end with ?" The virtual punctuation marks added by the 
prepaiser indicate that before the quote there is something which can be either a question mark or 
nothing at all. After the quote there is something that can be either a period or a question mark. 
The grammatical structure of the rest of the sentence allows later processing stages to select the 
5 best choice. 

The preparser may also use several further approaches in preparsmg a period (stqjs 130, 
132, 134, 136, and 138). Some abbreviations in the dictionary are marked as nevo- beginning 
sentences and others as never ending sentences (step 1 30). These rules are always respected. 
For example, "Ltd" never begins a sentence and "Mr" never ends one. The preparser also will 

1 0 not end a sentence with a single initial followed by a period unless the next word is a common 
grammatical word (step 1 32) such as "the", "in", etc. If the word before the period is found in 
any dictionary, the period will end the sentence (step 134). If the word before the period is not in 
this dictionary, and it has internal periods (e.g., I.B.M.) and the next word is not in the dictionary 
in a lowercase form, or the word after that is itself uppercase, then this is not an end of sentence 

1 5 (step 1 36). In remaining cases the period does mark the end of sentence (step 138). 

Referring again to FIGS. 2 and 3, once the sentence boundaries have been defined by the 
preparser, the parser places the words of the sentence in syntactic categories, and applies 
grammar rules ftom the grammar database to them to compute possible syntactic interpretations 
25 of the sentence (step 1 04). These grammar rules 48 can be implemented as a series of 

20 computer readable rules that express the grammatical constraints of the language. For tiie 
- English language, there may be hundreds of such rules, which may apply to hundreds of 
syntactic categories. To reduce the computational overhead of this operatipn, the different 
possible meanings of a word are ignored. 

In the next step (step 106), the graph maker employs the dictionary to expand the results 

25 of the parser to include the different meanings of words and creates a directed acyclic graph 

representing all semantic interpretations of the sentence. This graph is generated with the help of 
a series of semantic propagation procedures, which are described below. These procedures 
operate on a series of authored grammar rules and, in some cases, access a semantic feature tree 
for semantic information. The semantic feature tree is a tree structure that includes semantic 

30 categories. It is roughly organized from the abstract to the specific, and permits the procedures 
to detemiine how semantically related a pair of terms are, both in terms of their separation in the 
tree and their levels in the tree. For example, "cat" and "dog" are more related than "cat" and 
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"pudding", and hence the former pair would be separated by a smaller distance within the tree. 
"Animal" and "cat" are examples of words that are stored at different levels in the tree, as 
"animal" is a more abstract term than "cat." 

Referring to FIG. 9, the graph includes nodes 80 and their subnodes 82, 84. 86 linked by 
5 pointers 88, 89, 90, 91 in a manner that indicates various types of relationships. A first type of 
relationship in the graph is one where nodes representing phrases possess pointers to constituent 
word nodes or sub-phrase nodes. For example, a node 84 representing the phrase "the bank" will 
be linked by pointers 92, 93 to the constituent words "the" 94, and "bank" 95. A second type of 
relationship in the graph is where phrase interpretations possess pointers to alternate ways of 

10 making the same higher-level constituent from lower-level pieces. For example, a node 80 

representing the phrase "by the bank" can have two source interpretation locations 81, 83, which 
each include pointers 88 & 89, 90 & 91 to their respective constituents. In this example, the 
different constituents would include different subnodes 84, 86 that each represent different 
meanings for the phrase "the bank". The structure of the graph is defined by the results of the 

1 5 parsing operation and is constrained by the syntax of the source sentence. The nodes of the 
graph are associated with storage locations for semantic information, which can be filled in 
during the process of semantic propagation. 

The semantic propagation part of the system operates to propagate semantic information 
from smaller constituents to the larger constituents they comprise. It applies to four classes of 

20 the syntactic categories used in the earher parsing operation: SEMNP (which includes noun-Uke 
. - . objects and prepositional phrases), SEMVP (verb phrase like objects, which usually take 
subjects), SEMADJ (adjectives) and VERB (lexical verb-like verbs that often take objects). 
Other syntactic categories are ignored within a rule. The granmiar rule author may also override 
the imphcit behavior below by specific maridngs on rules. These specific instructions are 

25 followed first. 

There are two aspects to the manner in which semantic features are propagated through 
the system. The first is a set of rules that tell from examining the noun-like and verb-like 
constituents in a grammar rule, which selectional restriction slots of the verb-like constiments 
apply to which noun-like objects. For instance, the rule for the verb phrase of the sentence: "I 
30 persuaded him to go" is roughly VP = VTl 1 + NP + VP (where VP is a verb phrase, VTl 1 is a 
type of transitive verb, and NP is a noun phrase). One exemplary default rule indicates that when 
a verb takes objects, selectional restrictions are to be applied to the first NP encountered to the 



wo 98/57271 



PCT/US98/00729 



- 17- 

right of the verb. Another rule says that VP restrictions on their subjects should be applied to the 
first NP foxmd to the left of a VP. Together these rules make sure that "persuade him" and "him 
go" are both evaluated for their semantic plausibility. As mentioned before, these rules reflect 
the complex grammar of the English language and there may therefore be quite a few of them. 

Referring to FIG. 6, the semantic propagation operation includes copying of selectional 
restrictions from SEMVPs to imperative sentences (step 140). If a SEMNP is being used as a 
locative expression, its goodness is evaluated against semantic constants defming good locations 
(step 142). If a rule involves a conjunction of two SEMNPs (detected because of ANDing 
together of syntactic features), the graph maker ANDs together the semantic features and applies 
the semantic distance expert (step 144). 

If, in the course of examining the rules specified for syntactic feature propagation, the 
graph maker locates a "head" SEMNP which gets propagated to a higher level (e.g., it becomes 
part of a SEMNP that includes more words), it propagates semantic features as well (step 146). 
However, if the "head" is a partitive word (e.g., "portion," "part"), it propagates from a SEMNP 
to the left or right instead. SEMVPs and SEMADJs are propagated in the same way, with the 
only exception being that SEMVPs and SEMADJs do not have any partitive situations (step 
1 48). Adjectives are part of the SEMVP class for this purpose. 

When a SEMVP is made fi-om a nile including VERBs, the graph maker propagates 
upward the VERB'S subject restriction unless the VP is a passive construction, in v**ich case the 
VERB'S first object restriction is propagated instead (step 150). In any rule containing SEMVPs, 
it attempts to apply the selectional restrictions of the SEMVPs to NPs encountered moving 
leftward from the SEMVP (step 152). In any rule containing SEMADJs, the graph maker 
attempts to apply the selectional restriction of the SEMADJ first to any SEMNPs encountered 
moving to the right from the SEMADJ, and if that faUs, tries moving to the left (step 154). 

For any remaining unused object selectional restrictions of a VERB (that have not been 
propagated upward because of passives), the graph maker applies them in turn to SEMNPs 
encountered in order to the right of the VERB (step 1 56). In all of these rales, a verb selectional 
restriction is used up as soon as it applies to something. In all rules up to this one, SEMNPs are 
not used up when something applies to them. Starting at this rale, the SEMNP does get "used 
up". Finally, if a rale makes a SEMVP. the graph maker determines if there are any SEMVPs or 
SEMADJs in it that have not yet been used, and if so, propagates them upward (step 158). 
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The system also performs feature matching oflinguistic features. Linguistic features are 
properties of words and other constituents. Syntactic feature matching is used by the parser, and 
semantic feature matching is used by the graph maker. But the same techniques are used for 
both. For instance, "they" has the syntactic feature plural, while "he" has the feature of singular. 
Feature matching uses marking on grammar rules so that they only apply if the features of the 
words they are to apply to meet certain conditions. For example, one rule might be: 

S = NP{@}+VP{@> 

where the @ signs mean that the number features of the NP and VP must match. So while this 
rule will allow "they are" and "he is", it will not allow "they is" and "he are". 

Feature match restrictions are broken into "local" and "long distance". The long distance 
actions may be computed when the grammar is compiled, rather than when actually processing a 
sentence. The sequence of long distance operations that must be performed is then encoded in a 

series of instruction bytes. 

The computation of long distance feature operations must start with an A-ary rule (i.e., 
one that may have more than two inputs on its right). The system then distributes codes to 
various binary rules so that feature sets end up being propagated between rules in the correct 
fashion. By breaking the n-ary rules into binary rules, the parsing operations are greatly 
simplified, but because the system keeps track of feature sets across binary rules, it retains the 
power of the long distance operations. 

The system of the invention also allows multiword "idioms" as part of the dictionary, 
whUe retaining representations of the individual words of which they are composed. These two 
forms may ultimately compete against each other to be the best representation. For instance 
"black sheep" is found in the dictionary with the meaning of a disfavored person. But in some 
cases the words "black sheep" may refer to a sheep which is black. Because both of the forms, 
are retained, this non-idiomatic usage may still be chosen as the correct translation. 

The idioms may belong to further categorizations. For example, the system may use the 
following three types: 

Almighty: United States of America 
Preferential: long ago 
Normal: black sheep 
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Almighty idioms suppress any other possible interpretation of any of the words that make up the 
sequence. Preferential idioms suppress other constituents of the same general type and that use 
the very same words. Normal idioms compete on an even footing with other entries. 

The resulting graph is to be evaluated by experts (step 108, FIG. 3), which provide scores 
that express the Ukelihood of correctness of interpretations in the graph. The system of the 
invention includes a scoring method that applies to aU partial sentences of any length, not just 
full sentences. An important element in the use of a graph is that a subtree is fully scored and 
analyzed only oiice, even though it may appear in a great many sentences. For example, in the 
phrase "Near the bank there is a bank.", the phrase "Near the bank" has at least two meanings, 
but the best interpretation of that phrase is deteimined only once. The phrase "there is a bank" 
similarly has two interpretations, but the best of those two is determined only once. There are 
therefore four sentence interpretations, but the subphrases are scored just once. Another feature 
of the graph is that each node is labeled with easily accessible information about the length of 
that piece of the sentence. This allows the best N interpretations of any substring of the EngUsh 
sentence to be found without reanalyzing the sentence. 

Although, in one implementation, only the N overall best analyses of the sentence are 
avaUable at any one time (N being a number on the order of 20), the use of a graph allows the 
system to integrate the result of a user choice about a smaller constituent and give a different N 
best analyses that respect the user's choice. Because all this is done vrithout reparsing the 
sentence or rescoring any substrings, it may be done quickly. 

... Referring to FIG. 8, operation of the expert evaluator 30 is based on-various factors that 
characterize each translation, which are handled by the various experts. The rule probability 
expert 170 evaluates the average relative firequency of grammar rules used to obtain the initial 
source language parse tree. The selectional restriction expert 1 78 evaluates the degree of 
semantic accord of the given translation. The dictionary entry probability expert 172 evaluates 
the average relative frequency of particular "parts of speech" of the words in the sentence used to 
obtain the initial source language parse tree. The statistics expert evaluates the average relative 
frequency of particular paraphrases chosen for the given translation. 

The system automatically determines the English "part of speech" (POS) for various 
individual English words. English phrases, and groups of English words. The system makes the 
automatic determination of the POS when translating sentences, and the system usually makes 
the correct choice. Occasionally, however, the sentence being translated is itself ambiguous. A 
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word or phrase that can be interpreted as more than one POS leads to several distinct but 
"correct" meanings for the sentence in which the word or phrase appears. It is possible for an 
operator of the system to override the system' s automatic POS determination and instead 
manually set the POS for any word, phrase, or group of words. For example, in the source 
English sentence "John saw a boy with a telescope", an operator of the system can set "a boy 
with a telescope" as a Noun Phrase to force the system to interpret the sentence to mean that the 
boy was carrying a telescope and thus reject the interpretation that John used a telescope to see 
the boy. An operator can address the situation where overriding the system's POS rules yields 
worse, not better, translation results by applying a few manual POS settings as possible or by 
applying less restrictive manual POS settings. Noun Phrase is less restrictive than Noun, and 
Group is the least restrictive POS setting. The following is a list of the various possible POS 
settings. 

PART OF sp r.rrH fpos^ 

Noun 

Noun Phrase 

Verb (transitive or intransitive) 

Verb Phrase 

Adjective 

Adjectival Phrase 

Adverb 

Adverbial Phrase 

Preposition 

Pr^>ositional Phrase ' 

Conjunction 

Group 

English 

The parts of speech "Adjectival Phrase" and "Adverbial Phrase" are useful in the situation where 
an English sentence may have a different meaning depending on how a particular prepositional 
phrase is interpreted by the system. For example, the sentence "We need a book on the fourth of 
July" means "We need a book about the American fourth of July hoUday" if "on the fourth of 
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July" has an adjectival interpretation, but the sentence means "On the fourth day of July, we need 
a book" if the phrase "on the fourth of July" has an adverbial interpretation. If the operator 
believes the system has automatically assigned the incorrect POS to "on the fourth of J.uly", the 
operator can manually set a different POS to "on the fourth of July" in the sentence "We need a 
5 book on the fourth of July". If an operator does not want the system to translate a particular 
v^ord, phrase, or group of words from English to Japanese, the operator can assign the POS 
"English" to the desired word(s), phrase(s), and/or group(s) of words. It also is possible for an 
operator to remove one or more POS settings, regardless whether the setdngs were assigned 
automatically by the system or manually by an operator. 

10 The system keeps track of statistical information from translation usage at each customer 

site at more than one level. For example, the system may maintain statistical counts at the 
surface form level (hov^ often was "leaving" used as a transitive versus an intransitive verb), and 
also at the meaning level (did it mean "leave behind" or "depart" from), and this second type is 
summed over occurrences of "leave", "leaves", "left", and "leaving". The system may also keep 

1 5 statistical counts separately for uses that have occurred within the last several sentences, and uses 
that have occurred at any time at the customer site. Furthermore, the system may distinguish 
cases where the user intervened to indicate that a particular word sense should be used, from 
cases where the system used a particular word sense without any confirmation from the user. 

The structural balance expert 182 is based on a characteristic of English and many other 

20 European languages pertaining to the lengths of constituents in a given sentence. In some (but 
not all) constructions, sentences which involve heavy (lengthy) elements to the left of light 
elements are disliked in these languages. For example: 

Mary hit Bill with a broom. 
Light Heavy 
25 {acceptable} 

Mary hit with a broom Bill. 

Heavy Light 
{unacceptable} 

Mary hit with a broom a dog that tried to bite her. 
30 Heavy Heavier 

{acceptable} 
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Given two parses of a given sentence, if one contains a "Heavy - Light" sequence involvii^ a 
construction that tends to avoid such a sequence, and if the other parse does not, then it can be 
assumed that the former does not represent the intended interpretation of the sentence. This 
expert is an effective way to distinguish between intended parses and unintended parses. 
5 In coordinate structures of the pattern of "A of B and C" it can be difficult to detennine 

whether the intended interpretation is "A of {B and C}" or "A {of B) and C'\ The coordinate 
structure expert 1 80 measures the semantic distance between B and C, and that between A and C 
to determine which mode of coordination combines two elements that are closer in meaning. 
This expert accesses the semantic feature tree during its operation. This expert is also an 
1 0 efficient way to distinguish between the intended parses and the unintended parses of a given 
sentence. 

Many words in English include potential ambiguities between ordinary-noun and proper- 
name interpretations. The capitalization expert 176 uses the location of capitalization in a 
sentence to determine how likely it is that the capitalization is significant For example, the 
15 following sentences: 

Brovm is my first choice. 
My first choice is Brown. 

are different in that while tiie former is genuinely ambiguous, it is far more likely in the latter 
that "Brown" is a person name than a color name. This expert takes into consideration factors 

20 such as whether a given capitalized word appears at sentence-initial or sentence-nonirutial 

position (as shown above), whetiier ttie capitalized spelling is in the dictionary, and whether the 
lower-case-initial version is in the dictionary. This expert is an effective way to obtain tiie 
correct interpretations of capitalized words in sentences. 

If a sentence contains a sequence of initial-uppercase words, it can be treated as a proper 

25 name or as a sequence of ordinary nouns. The system of the invention employs a capitalized 
sequence procedure, which favors the former interpretation. Also, if the sequence carmot itself 
be parsed by normal grammar rules, it can be treated as a single unanalyzed noun phrase to be 
passed through untranslated. This procedure has proven to be a very effective way of dealing 
with multiple- word proper names vy^iile not completely ignoring the lower-rated common noun 

30 interpretations. 
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Refeiiing to FIG. 7, the machine translation system of the invention uses a grammar-rule 
controlled structural conversion mechanism 162 that has the efficiency of a straightforward 
grammar-nile-based structuial conversion method, but which comes close to the power of the 
template-to-template structural conversion method. This method relies on the use of grammar 
rules 160 which can specify non-flat complex substructure. While the following is a rule format 
used in other translation systons: 

Y => XI + X2 + ... Xn 
Substructure Specified 



XI X2... Xn 
the system of the invention uses grammar rules of the following format: 
Y -> #Z1 (i) #Z2(2) XI + X2 ... + Xi + X(i+1) + ... X(n) 
Substructure Specified 
Y 

/ I \ 

#Z1 X(i+1) Xn 

/ I \ 
#S2 Xi X(i+1) 

/ \ 
XI X2 

In this syntax, symbols prefixed with are virtual symbols that are invisible for the purpose of 
sentence structure parsing, but which are used in building substructures once a given parse is 
obtained. 

Given this type of grammar, it becomes possible to specify multiple structure conversion 
among any sequences of nodes that are siblings in the substructure. This transforms a grammar- 
rule-based stracture conversion mechanism into one with some of Ae power of the template-to- 
template structure conversion mechanism. Although the system of the invention is based on the 
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second type of grammar rules presented above, it automatically compiles the corresponding 
grammar rules of the first form. It can therefore use grammar rules of the first form to parse 
sentences, and use grammar rules of the second form to build parse structures. 

The structural conversion also includes a dictionary controlled structural conversion 
operation 166. which accesses dictionaries 161 to operate on the parse tree after it has been 
operated upon by the grammar-rule controlled structural conversion operation. The synthesis- 
rule controlled structural conversion operation then applies synthesis rules to the resulting parse 
tree to provide the target lai^uage text 41. 

Referring again to FIGS. 1 and 2, after the system has derived a best-ranked translation in 
the process described above, it is presented to the user via the display 20. The user then has the 
option of approving or editing the translation by interacting with the alternate parse system 37, 
via the user input device 22. In the editing operation, the user may constrain portions of the 
translated sentence that are correctly translated, while requesting retranslation of the remaining 
portions of the sentence. This operation may be performed rapidly, since the system retains the 
graph with expert weights 31. 

Having described certain details of the automated natural language translation system 
with reference to FIGS. 1-9, improvements according to the invention are now described with 
reference to FIGS. lOA through 1 6. These improvements automatically translate input text vs*ile 
the text is being generated by a user, operator or system, such as, for example, while a user is 
generating a document using word processing software. An advantage to the system of the 
present invention lies in the ability of the system to translate input text as it is being generated or 
shortly after the text is generated, which can be usefiil in a language learning, as well as m 
systems generating temporal text that must be translated quickly and efficiently. 

In accordance with the present invention, a user, operator or system, can enter, generate, 
create, edit, receive or otherwise compile a text file or document, (hereinafter interchangeably 
referred to as "generate a text file" or "generate a document"), by entering or adding to a 
computerized system or file, characters, symbols and text, (hereinafter interchangeably referred 
to as characters, character stream, textual input or input text stream), through the user input 
interface 22, or the input interface 12, as shown in Fig. 1. A document or text file can include 
any number of characters, formats, texts or files, depending on the desired application. As 
described above, the user interface 22 and the input interface 12 can comprise, for example, a 
keyboard, mouse, touchscreen, light-pen, voice-activated transmitter, or other input device, as 
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well as a digital interface such as a modem or serial port It is to be appreciated that the system 
of the present invention can incorporate or interface with word processing software, such as, for 
example, Microsoft® Word 6.0, installed on the user's workstation (20, 22). Thus, as the user is 
generating a document in a source language, a natural language translation of the document to 

5 the target language is automatically carried out, without the user having to affirmatively request 
translation of the input text, such as, for example, by clicking on a translation icon. The 
translation system of the present invention can, however, accommodate affirmative translation 
requests. The system of the present invention can further interface with other systems in the 
event that the document to be translated is generated by a computerized system. It is to be 

1 0 appreciated that a document can be generated using the user input interface 22 and the input 
interface 1 2, either independently, or in combination. 

Referring to Fig. 1 OA in conjunction with Fig. 1 , shown is a functional block diagram 
illustrating the system for automated translation and retranslation according to one embodiment 
of the present invention. As shown, input text 183 generated or entered into the system through 

1 5 the user input interface 22 and/or the input interface 1 2 is sent to the translation engine 1 6 in 
substantially real time, where it is translated 1 84 from the beginning of the input text in the text 
file or document being created. For example, the translation can commence with the first 
character in a stream of characters entered, the first word in a stream of words and characters 
entered, or other starting point, as will be fiirther described. The input text 1 83 is also stored 1 86 

20 as a text file in storage 1 8 as it is being generated or entered, for subsequent retranslation 1 84 and 
display 1 85 . The stored input text 1 86 can be used in the translation and retranslation 184 of the 
text file, such that the input text immediately received, as well as oflier previously entered input 
text in the existing text file, is translated. The translation and the input text are then displayed 
1 85 and continuously refreshed as the existing text changes with newly received input text. 

25 The process of translating and retranslating 1 84 the input text is shown as a loop that 

constantly translates the input text 1 83 as it is being received, transmits as output, translated text 
to the display 185, and loops back to translate 1 84 any newly received input text along with any 
previously received input text from storage 1 86. If no new input text has been received, the 
translation process is temporarily suspended until input text is again received. Therefore, in 

30 substantially real time, the input text is translated along with input text in the existing text file, 
and both the input text in the souree language, and the output text in the target language can be 
displayed. It is to be appreciated however, that in other embodiments, only the translated text 
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need be displayed, and the input text in the source language can be obtained by clicking on a 
translated word in the document, or fix>m a system clipboard. 

Referring to Fig. lOB, shown is a functional block diagram illustrating the system for 
automatic translation and retranslation according to another embodiment of the present 
5 invention. As similarly described above, input text 1 83 generated or entered into the system 
through the user input interface 22 and/or the input interface 12, is sent to the translation engine 
1 6, and stored 1 86 as a text file in storage 1 8, in substantially real tune. The input text is then 
compared 1 87 to the text file up to the point that the input text was received, to determine which 
of the input text is newly received text that has not previously been translated. After a 

1 0 determination is made as which of the input text is newly received ii^jut text, such as, for 

example, the entry of the words comprising the continuation of a sentence or a modification of a 
previous sentence, a translation 1 88 of the newly received input text is carried out. Depending 
on where the newly received input text appears, a retranslation 188 of previously received input 
text can also be carried out, such as, in the above example, where the newly received input text 

1 5 modifies a previously entered sentence. In this manner, the stored ii^}ut text 186 can be used in 
the translation and retranslation 1 84 of the text file, such that the input text immediately received, 
as well as other previously entered input text are translated. Both the translated text and the 
input text can then be displayed 1 85. The process of translating and retranslating 188 the input 
text is shown as a loop that, in substantially real time, translates the newly received input text, 

20 transmits as output the translated text to the display, and then loops back to compare 1 87 the 

newly received input text with the text file and translate any newly received input text.. If no new 
input text has been received, the translation process can be temporarily suspended, as described 
above. 

Referring again to Fig. 1 , the translation activator 21 included in the translation engine 16 
25 determines when the translation of the document is to be initiated, that is, when the translation 

engine 1 4 is to effect a translation of the document from the source language to a target language. 
Referring to Fig. 1 1 , shown in further detail is the translation activator 2 1 according to one 
embodiment of the invention. As shown, the translation activator 21 can comprise a processing 
module 200 incorporating an oscillator, timer or counter 202 (hereinafter referred to as a "counter 
30 202") that senses when no signal is present at an input 204 fi-om the user input device 22 or the 
input interface 12. For example, as a user is typing via a touchscreen 190, keyboard 192, or 
mouse 1 94, one or more of the inputs 204 to the counter 202 is active and the counter 202 is off, 
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that is, it is not counting. When no inputs 204 are active, or when only certain of the inputs 204 
are active, the counter 202 begins counting from a starting value such as zero. The counter 202 
provides to a comparator 206, an output representing its count, which compares the count to a 
predetennined value 208. The predetermined value relates to time and is preferably a number 
stored in storage 1 8. For purposes of illustration only, when the translation system of the 
present invention is used for translating text as a user types it into a word processing document, 
the value 208 can represent a time period from about . 1 second to about 45 seconds, which is 
about equivalent to the length of time the user might pause to think about what he or she wishes 
to type next, or to scroll through the document, for exampic. Additionally, in another 
embodiment of the invention, where the translation system of the present invention is used for 
translating natural language text from a computer generated source, the value 208 can represent a 
shorter or longer time period. It is to be appreciated that the value 208 can correspond to any 
time period suitable to a desired use of the system of die present invention. 

When the count from the counter 202 is substantially equivalent to or otherwise 
corresponds to the value 208, the comparator 206 provides a signal to the translation engine 16 
causing the translation process to begin. Where the count is not equivalent to or does not 
correspond to the predetermined value 208, the comparator 206 provides a signal to an adder 
210, causing incrementation of the counter 202. Thus, should there exist a delay in receiving a 
textual input from the user input device 22 or the input interface 12, the counter 202 counts until 
the count substantially equals a predetermined value 208, at which point, a translation occurs. It 
is important to note, however, that certain actions such as depressing an enter -key or clicking, on 
a translation icon, can still cause a translation to occur, as shown by the inputs 196 to die 
translation engine 16 from the touchscreen 190, keypad 192 and mouse 194. 

Referring to Fig. 12A, shown is a flow chart illustrating the process for translating textual 
input as a document is being generated, according to one embodiment of the invention. As 
shown in step 220, the translation activator 21 waits for a lapse in the receipt of an input 
character stream from the user input device 22 or the input interface 12. Upon sensing a lapse, 
the counter 202 begins counting from an initial value 208, as shown in step 222. Step 224 is then 
executed, and the translation activator 21 then senses either actively or passively, the existence of 
a keyboaid stroke, which is not to be limited herein to refer to only a keyboard input, but rather 
can comprise any input from a user device 22 or an input interface 12. using one or more of the 
components described above. (i.e. keyboard, mouse, touchscreen, light pen, voice-activated 
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transmitter) and those devices otherwise used as input or signaling devices. If the output of step 
224 is affirmative, step 220 is executed, and the translation activator 21 agam vvaits to sense a 
lapse in the input character stream. In the event that a keyboard stroke is not sensed, step 226 is 
executed and the translation activator 21 determines whether the count has reached a 
5 predetermined value 208. If the count is less than the predetermined value 208, step 228 is 
executed, where the counter 202 is incremented, and steps 224 and 226 are again repeated. 

In the event that the counter 202 has reached the predetermined value 208, translation of 
the source document can commence at a predefined or starting location in the document or file, 
such as, for example, the beginning of the document or file. In the present embodunent, the 
1 0 beginning of the document can be designated by the first character entered or added by the user, 
or by the computer system, in generating the document. In other embodiments, depending on the 
desired objectives, the translation can begin with the first character or word entered or added in 
the most recently entered sentence, paragraph or page. For example, where a user has typed into 
a document or file, a two page memo, the translation can begin with the first character on page 

1 5 one of the memo. 

Referring to Fig, 12B, shown is a flow chart illustrating the process for translating textual 
input as a document is being generated, according another embodiment of the invention. As 
similarly described above, the translation activator 21 waits for a lapse 220 in the receipt of an 
input character stream, and upon sensing a l^se, begins countmg 222 unless interrupted by a 

20 keyboard stroke 224. When the count reaches a predetermined value, an existing text file 

comprising previously received input text is compared 227 to an updated text file that includes 

immediately received input text, and a determination is made as to that which comprises the 
immediately received text. As described above in Fig. lOB, input text is stored as a text file and 
translated as it is received. When input text is received thereafter, a determination is made by 

25 examining the previously stored, existing text file and comparing it with the immediately 
received input text. In this manner, only the immediately received text, or the inunediately 
received text and any other text necessary, is then translated in step 229. 

Referring to Figs. 13 A and 13B, shown is an example of a memo being translated as it is 
generated, using the natural translation system of the present invention. As shovsm in Fig. 1 3 A, 

30 as the user types in a character stream 250 through the user mterface, the words "The revised 
building permit", a pause is sensed after "permit", during which the user is not entering any 
characters, commonly due to a pause that one takes in thinking about what he/she wishes to type 
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next, or a computer-generated pause. At this point, the translation activator 21 determines that a 
pause equivalent to a predetermined time period, such as, for example, 3 seconds has elapsed, 
and a translation of the English words into Japanese is carried out from the beginning of the 
document using the translation engine. The translation is then presented to the user via the 
display 20. It is to be appreciated that the translation can further be presented in hard copy 
through a printer, or as an audio signal, such as words spoken through a voice-simulation 
transmitter. As shown, the system can be very useful for language learning, as the input text is 
presented alongside the translated text, allowing the user to view the output text translation as 
he/she is creating the input text. 

Referring now to Fig. 1 3B, as the user resumes typing, the character stream 252 is sensed 
with each keyboard stroke, and tiius the translation activator 21 returns to a waiting mode. The 
translation engine then senses a pause after tiie user has typed in ''will be", causing the 
translation to commence -with the initially entered characters, to wit, "From: Richard Stevens. . ." 
It is to be ^preciated that in other embodiments, the translation can conmience from the 
beginning of tiie sentence, raflier tiian tiie beginning of the document, as described above. Thus, 
the translation would commence with the words "The revised. . . ". and continue to tiie word 
"permit." 

It is important to note that by starting the translation at tiie beginning of die document, 
that is, by retranslating die document, tiie accuracy of the interaiediary as well as tiie final 
translation can be enhanced, as the translation engine is able to capture any changes in the 
current sentence structure that would affect tiie translation of previous .words in the sentsaice,. .. . 
such as, for example, subsequent changes that may affect tiie verb. Additionally, where changes 
have been made to preceding words, sentences or paragraphs, such as, for example, edits 
involving cutting and pasting functions, the translation engine can ensure that the translation into 
the target document includes such edits. To illustrate a translation in which changes to 
subsequent words in a sentence affects tiie translation of preceding words, consider, the 
following example of a French translation that compares the translation of a sentence fragment 
witii tfie translation of the entire sentence after additional words are added to the sentence 
fragment: 
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Entryl: 
ENGLISH: lam 
FRENCH: Jesuis 

F-ntrvTTi 

5 ENGLISH: I am not 

FRENCH: Je ne suis pas 
Thus, if the translation were started from where the first sentence left off, that is, if the translation 
of the second entry was started after "not", the French translation would not be accurate, as it 
would not reflect the changes that are requned to be made in the sentence's verb structure in the 

10 French language. 

Referring to Fig. 1 4, shown is another embodiment of the process for automatically 
translating textual input. As similarly described above, the translation activator 21 senses a lapse 
and begins counting from an initial value 208, as set forth in steps 320, and 322. In this 
embodiment in step 324, when a keyboard stroke is sensed, step 325 is executed, and a 

1 5 determination is made as to whether the keyboard stroke represents an enter fimction, mouse 
click, pressing a touchscreen, or other input signal that represents a pause, or function that does 
not relate to entry of an input character stream. In this embodiment, the depression of an enter 
key can represent a pause in generating an input character stream, for example, when a user has 
come to the end of a paragraph and wishes to add a space between the last paragraph and a new 

20 paragraph. Similarly, the cUcking of a mouse can represent a pause in generating an input 

character stream, for example, when a user clicks on an icon to underline the text which follows, 
or when a user clicks on text to perform a drag and drop fiinction. Such actions can be 
interpreted by the translation activator as allowing enough time for the translation of the input 
text to be carried out. That is, in such examples, the translation activator 21 considers the 

25 keyboard strokes and mouse clicks as being similar to a pause, and thus causes a translation to 
occur, as shown in step 330, from the beginning of the document to the point where the user has 
stopped, such as for example, the last word or character typed into the document. Alternatively, 
as described above, only immediately received text can be translated. 

It is important to note that in another embodiment of the invention, the translation 

30 activator 2 1 can be configured such that the flow chart of Fig. 1 4 senses an affirmative request 
for translation. That is, in step 325, the system can determine whether the user, operator or 
system is requesting translation of the document by depressing an enter key, clicking on an icon. 
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pressing a touchscreen or otherwise generating an input signal indicative of a translation request. 
In such an embodiment, a keyboard stroke or a click on a translation icon results in automatic 
translation and retranslation of the text in the document. 

Referring again to step 324, where the keyboard stroke sensed is other than an enter, 
mouse cHck or other input such as those described above, step 326 is executed and a 
determination can be made regarding whether the counter 202 has reached a predetermined value 
208, in which case, step^30 is executed and a translation of the document is generated from a 
starting point. If the counter 202 has not reached the predetermined value, step 328 is executed, 
the counter is incremented, and steps 324 and 326 are again repeated. 

Referring to Fig. 15, shown is another embodiment of the invention, in which the 
translation activator 21 senses the formation of a sentence prior to initiating a translation of the 
input character stream. As similarly described above in step 420, the translation engine 16 is 
receiving an input character stream and the translation activator 21 senses a pause. In step 422, 
the translation activator 21 examines the character stream to determine whether a sentence or 
sentence fragment, which can include, for example, a character, a sequence of characters, a word, 
or a sequence of words, arc formed by the input character stream. In the present embodiment, if 
a sentence or sentence fragment has not yet been formed, an input character stream is awaited in 
step 423. Once an input character stream is again sensed, step 420 is executed, where the 
translation activator 2 1 vraits for a subsequent lapse in the input character stream. If a 
determination is made in step 422, that a sentence or sentence fragment has formed, step 424 is 
executed, in which a determination is made either actively or passively, as to whether a keyboard 
stroke is sensed. If a keyboard stroke is not sensed, step 430 is executed and the document is 
translated from a starting point. If a keyboard stroke is sensed in step 424, step 420 is executed, 
where the translation activator 21 again waits for a lapse in the input character stream. As shown 
in dotted lines, the translation activator 21 can optionally determine in step 425, whether an input 
has been received, such as an enter or a mouse click, representing a pause or a function that does 
not relate to entry of an input character stream, as described above in Fig. 14. As similarly 
described above, sensing such an input causes step 430 to be executed, where a translation is 
carried out from a starting point in the document, such as, for example, the beginning of the 
document, or the beginning character of the immediately received text. 

It is to be appreciated that in other embodiments, the translation activator 21 of the 
present invention can, in step 422, determine whether a recognizable word in a source language 
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has been generated. In this embodiment, a recognizable word, such as, for example, the English 
word "an" as opposed to "av", would result in an affirmative determination and passage to step 
424. In the case of a non-recognizable word, such as "av", the translation activator 21 can wait 
for additional characters in step 423 to determine whether a word has been generated, such as, for 
example, where the user continues to type, creating the word "avenue". Alternatively, the 
translation activator can determine in step 422, whether characters or words generated after the 
non-recognizable character(s) or word are recognizable, such as. for example, in the case of a 
typographical eiror followed by correctly entered characters or words. In another embodiment, 
the translation activator 21 can determine, in step 424, whether a paragraph has been formed by 
the input character stream. In such embodiments, the translation engine 14 would execute a 
translation operation only after an input character stream generates a paragraph. 

Referring to Fig- 13C, shovra is an example of the translation system of the present 
invention, in which the translation activator 21 is configured to sense, during a lapse in typing, 
whether complete sentences has been typed, and translate only complete sentences. As shown in 
this embodiment, the sentences comprising paragraph 254 are complete sentences and are 
translated. Note however, that the sentence fragment 256 "This should not delay the, . ." is not 
yet translated, as a determination is made by the translation engine 1 6 that these words merely 
comprise a sentence fragment. 

Referring now to Fig. 1 6, shown is an embodiment of the present invention that combines 
the concepts of, determining whether a predetermined count value 208 has been reached, as well 
as determining whether a sentence has been formed by the input character stream. As similarly, 
described above, the translation activator 21 waits, in step 520. for a lapse in input character 
stream, and begins counting in step 522 from an initial value 208. The translation activator 21 
then determines in step 524 whether a keyboard stroke is passively or actively sensed, in which 
case, step 520 is again executed, and the translation activator 21 waits for a lapse in the input 
character stream. In the event that a keyboard stroke is not sensed, a determination is made in 
step 526 whether the counter 202 has reached a predetermined value 208. In the event that the 
counter 202 has not yet counted to such value 208, step 528 is again executed, and steps 524 and 
526 are again repealed. When it is determined that the counter 202 has reached the 
predetermined value 208, step 529 is executed, where the translation activator 21 determines 
whether the input character stream forms at least one sentence. In the event that the input 
character stream does not form a sentence, step 532 is executed, where an additional stream of 



wo 98/57271 



PCT/US98/00729 



-33- 

input text data is awaited. Once an additional input character stream is sraised, control is again 
routed to step 522, where the translation activator 2 1 waits for a subsequent lapse in the input 
character stream and steps 522 through 529 are again executed. If a determination is made in 
step 529, that the character stream forms a sentence, step 530 is executed, and the document is 
5 translated from a starting point. It is to be appreciated that in other embodiments of the invention 
step 529 can sense the formation of a word, sentence fragment, or paragraph prior to translating 
the input text 

The automatic translation and retranslation system of the present invention can be used as 
a language learning tool to enable a person Huent in a source language to learn a target language. 

1 0 For example, as illustrated in Figs. 1 3 A- 1 3C, a person fluent in English can easily use the 

translation system of the present invention to learn Japanese. As shown and described above, a 
user at a computer workstation having a display monitor, can, upon typing in a word or sequence 
of words, view on the display screen, an automatic translation of the typed words. Moreover, as 
the user continues to type, the user can view the changes that may occur in the translation of the 

15 preceding words due to grammatical and other structural influences. 

It is further important to note that the system of the present invention can be configured 
to accommodate translation of a source language into a plurality of target languages for the user 
to view together, such as for example, English to Japanese, and English to Korean; or Biglish to 
Japanese, and then Japanese to Korean. 

20 Additionally, the system of the present invention can be used to translate computer 

■ generated natural language text that is temporal in its relevance, applicability or fotmati such as, 
for example, an important or urgent message required to be made known to a number of network 
users in a number of different countries. In such an example, the system of the present invention 
can be used to translate the message into one or more languages as it is being generated, aUowing 

25 the message to quickly reach and be understood by the relevant users. 

The present invention can further be useful in Intemet related applications, such as instant 
messages, electronic mail, web pages, bulletin boards and Intemet chat rooms; to name for 
purposes of illustration, a few suitable applications. Is to be appreciated however, that numerous 
other applications are within the scope of the present invention. 

30 All of the above-described functions and operations may be implemented by a variety of 

hardwired logic design and/or programming techniques for use with a general purpose computer. 
The steps as presented in the flowcharts generally need not be applied in the order presented, and 



wo 98/57271 



PCT/US98/007a9 



-34- 

combinations of the steps may be combined. Similarly, the functionality of the system may be 
disuibuted into programs and data in various ways. Furthermore, it may be advantageous to 
develop the grammar and other operative rules in one or more high level languages, while 
supplying them to end xisers in a compiled format 
5 Any of the embodiments of the automated natural language translation system described 

herein, including all of the functionality described herein, can be provided as computer software 
on a computer-readable medium such as a diskette or an optical compact disc (CD) for execution 
on a general purpose computer (e.g., an Apple Macintosh, an IBM PC or compatible, a Sun 
Workstation, etc.), 

10 Variations, modifications, and other implementations of what is described herein will 

occur to those of ordinary skill in the art without departing from the spirit and the scope of the 
invention as claimed. Accordingly, the invention is to be defined not by the preceding 
illustrative description but instead by the spirit and scope of the followmg claims. 
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What is claimed is: 

Claims 

1 1 . An automated natural language translation system, comprising: 

2 a memory module; 

3 a receiving module in communication with the memory module, receiving input textual 

4 information in a source natural language and storing the input textual information in the memory 

5 module; 

6 a translation engine in communication with the receiving module, accessing the memory 

7 module and translating the input textual information in the source natural language into output 

8 textual information in a target natural language; and 

9 a processing module in communication v^dth the receiving module, determining when the 

10 receiving module has not received input textual information for a predetermined period of time 

1 1 and transmitting an instruction signal to the translation engine to translate the stored input textual 

12 information. 

1 2. The system of claim 1 , wherein the source natural language is English and the 

2 target natural language is Japanese. 

1 3 . The system of claim 1 , further comprising a user device in communication with 

2 the receiving module, providing input textual information in the source language. 

1 4. The system of claim 3, wherein the receiving module receives input textual 

2 information from the user device as a user is creating a document in the source language with the 

3 user device. 

1 5. The system of claim 3, wherein the user device comprises at least one of the 

2 following: a computer keyboard, a mouse, a touchscreen, or a voice-activated transmitter, 

1 6. The system of claim 5 wherein the processing module transmits the instruction 

2 signal when a determination is made that the receiving module has received a signal representing 

3 the depression of an enter key on the computer keyboard. 
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7. The system of claim 6, wherein the processing module transmits the instruction 
signal when a det^mination is made that the receiving module has received a signal representing 
the clicking of the mouse. 

8. The system of claim 1 , wherein the predetermined period of time is within the 
range of .3 seconds to 1 second. 

9. The system of claim 1, further comprising a display in communication with the 
translation engine, displa3ang the input textual information in the source language and the output 
textual information in the target language. 

1 0. The system of claim 1 , the processing module fiirther comprising a counter for 
incrementing a coxmt until the count equals the predetermined period. 

1 1. The system of claim 10, wherein the counter is reset when the receiving module 
receives input textual information. 

12. The system of claim 1 , wherein the translation engine translates input textual 
information that was received by the receiving module prior to a time of transmission of the 
instruction signal. 

13. The system of claim 1, wherein the input textual information comprises a stream 
of characters transmitted ftom an input device. 

14. The system of claim 1, wheiein the processing unit determines when the receiving 
module has not received textual infonnation by determining the existence of a time lapse in the 
input textual information equivalent to the predetermined period, prior to transmitting the 
instruction signal to the translation engine. 

15. A method of automatically performing a natural language translation of a 
chamcter stream, comprising: 

receiving from a user device, a character stream in a source language comprising a 
plurality of characters; 
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5 determiniBg whether a time lapse exists in the receipt of the character stream, 

6 during which time lapse the character stream is not received; 

7 translating the character stream from the source language to the target language 

8 when the time lapse exists. 

1 1 6. The method of claim 1 5, further comprising: 

2 detcraaining whether the time lapse is equivalent to a predetermined period; and 

3 translating the character stream from the source language to the target language 

4 beginning with a first character in the character stream, when the time lapse is equivalent to the 

5 predetermined period. 

1 1 7. The method of claim 1 6, wherein the predetermined period is within the range of 

2 about .3 seconds to about 2 seconds. 

1 1 8. The method of claim 1 5, wherein the character stream comprises at least one word 

2 in the source language. 

1 19. The method of claim 1 5, wherein the character stream comprises at least one 

2 sentence in the source language. 

1 20. The method of claim 1 5, further comprising : 

2 generating the character stream using a computer keyboard; and 

3 displaying on a computer monitor the character stream in the source language; and 

4 the translated character stream in the target language. 

1 21. In a language learning system, an automated natural language translation system 

2 comprising: 

3 a receiving module interfacing with a user workstation and receiving a character stream in 

4 a source language generated by a user at the user workstation; 

5 a processing module in communication with the receiving module, determining whether a 

6 time period has elapsed from the receipt of a character in the character stream; 
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7 a translation module in communication with the processing module, translating the 

8 character stream in the sotirce language to a character stream in the target language; and 

9 transmitting the translated character stream to the user workstation. 
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